Search for: All records

While semi‐autonomous drones are increasingly used for road infrastructure inspection, their insufficient ability to independently handle complex scenarios beyond initial job planning hinders their full potential. To address this, the paper proposes a human–drone collaborative inspection approach leveraging flexible surface electromyography (sEMG) for conveying inspectors' speech guidance to intelligent drones. Specifically, this paper contributes a new data set,sEMGCommands forPilotingDrones (sCPD), and ansEMG‐basedCross‐subjectClassificationNetwork (sXCNet), for both command keyword recognition and inspector identification. sXCNet acquires the desired functions and performance through a synergetic effort of sEMG signal processing, spatial‐temporal‐frequency deep feature extraction, and multitasking‐enabled cross‐subject representation learning. The cross‐subject design permits deploying one unified model across all authorized inspectors, eliminating the need for subject‐dependent models tailored to individual users. sXCNet achieves notable classification accuracies of 98.1% on the sCPD data set and 86.1% on the public Ninapro db1 data set, demonstrating strong potential for advancing sEMG‐enabled human–drone collaboration in road infrastructure inspection. 

Abstract Segmentation of structural components in infrastructure inspection images is crucial for automated and accurate condition assessment. While deep neural networks hold great potential for this task, existing methods typically require fully annotated ground truth masks, which are time‐consuming and labor‐intensive to create. This paper introducesScribble‐supervised StructuralComponent SegmentationNetwork (ScribCompNet), the first weakly‐supervised method requiring only scribble annotations for multiclass structural component segmentation. ScribCompNet features a dual‐branch architecture with higher‐resolution refinement to enhance fine detail detection. It extends supervision from labeled to unlabeled pixels through a combined objective function, incorporating scribble annotation, dynamic pseudo label, semantic context enhancement, and scale‐adaptive harmony losses. Experimental results show that ScribCompNet outperforms other scribble‐supervised methods and most fully‐supervised counterparts, achieving 90.19% mean intersection over union (mIoU) with an 80% reduction in labeling time. Further evaluations confirm the effectiveness of the novel designs and robust performance, even with lower‐quality scribble annotations. 

Traffic accident anticipation is a vital function of Automated Driving Systems (ADS) in providing a safety-guaranteed driving experience. An accident anticipation model aims to predict accidents promptly and accurately before they occur. Existing Artificial Intelligence (AI) models of accident anticipation lack a human-interpretable explanation of their decision making. Although these models perform well, they remain a black-box to the ADS users who find it to difficult to trust them. To this end, this paper presents a gated recurrent unit (GRU) network that learns spatio-temporal relational features for the early anticipation of traffic accidents from dashcam video data. A post-hoc attention mechanism named Grad-CAM (Gradient-weighted Class Activation Map) is integrated into the network to generate saliency maps as the visual explanation of the accident anticipation decision. An eye tracker captures human eye fixation points for generating human attention maps. The explainability of network-generated saliency maps is evaluated in comparison to human attention maps. Qualitative and quantitative results on a public crash data set confirm that the proposed explainable network can anticipate an accident on average 4.57 s before it occurs, with 94.02% average precision. Various post-hoc attention-based XAI methods are then evaluated and compared. This confirms that the Grad-CAM chosen by this study can generate high-quality, human-interpretable saliency maps (with 1.23 Normalized Scanpath Saliency) for explaining the crash anticipation decision. Importantly, results confirm that the proposed AI model, with a human-inspired design, can outperform humans in accident anticipation.